An Integrative Approach to Portfolio Evaluation for Teacher Licensure

The purpose of our overall research agenda is to develop and evaluate a methodology for the assessment of teachers in which experienced teachers, serving as judges, engage in dialogue to integrate multiple sources of evidence about a candidate to reach a sound conclusion. The project that provides the venue for this research agenda is the Interstate New Teacher Assessment and Support Consortium (INTASC), which is developing a portfolio assessment system to assist participating states in making a decision about teacher licensure. To develop the theoretical foundation necessary to support and evaluate such dialogic and integrative assessment practices, we turn, in part, to the tradition of philosophical hermeneutics, as a complement to psychometrics. In this article, we characterize and assess the processes in which judges, trying out an integrative approach to portfolio evaluation for the ®rst time, engage as they collaboratively construct and document their conclusions, and we locate this work in the larger research agenda. The premise of this project, which is being carefully evaluated in the course of inquiry, is that these integrative practices cannot only lead to an epistemologically sound evaluation of teaching but also promote an ongoing professional dialogue of critical re ̄ection on teaching practice. Current calls for reform in the professional development and practice of teachers consistently highlight particular themesÐthat successful teachers integrate complex evidence about their students' learning and the context in which they work, that they engage in ongoing critical re ̄ection about their own teaching practices, and that they work as members of active learning communities (e.g., Darling-Hammond, 1995; INTASC, 1992; Lieberman, 1990; Little, 1993; Lord, 1994; Meier, 1995; NBPTS, n.d.; National Commission on Teaching and America's Future, 1996; Richardson, 1990; Tyack & Cuban, 1995). Consistent with this reform effort is the movement toward performancebased licensing and certi®cation of teachers and the preparation of accomplished teachers to serve as judges for these assessments (INTASC, 1995; NBPTS, 1994). These are important steps toward bringing assessment practices in line with the goals of reform. As important as these steps are, there exists a substantial disjunction between the assessment practices teachers typically engage in as judges or readers in large-scale assessment and the goals of the current reform movement. While psychometric theory has advanced to accommodate the evaluation of complex performances (e.g., Cronbach, Linn, Brennan & Haertel, 1995; Wiley & Haertel, 1994; Mislevy, 1994), conventional procedures for evaluating these performance assessments typically have individual readers working independently, scoring one exercise at a time, blind to the candidate's performance on other exercises. To reach a decision about certi®cation, these independent scores are algorithmically combined and compared to a cut score predetermined by a Journal of Personnel Evaluation in Education 12:2 139±161, 1998 # 1998 Kluwer Academic Publishers, Boston ± Manufactured in The Netherlands